Combining Adversaries with Anti-adversaries in Training

Adversarial training is an effective learning technique to improve the robustness of deep neural networks. In this study, influence adversarial on models in terms fairness, robustness, and generalization theoretically investigated under more general perturbation scope that different samples can have directions (the anti-adversarial directions) varied bounds. Our theoretical explorations suggest combination adversaries anti-adversaries (samples with perturbations) be achieving better fairness between classes a tradeoff some typical scenarios (e.g., noisy label imbalance learning) compared standard training. On basis our findings, objective combines bounds each sample presented. Meta utilized optimize weights. Experiments benchmark datasets verify findings effectiveness proposed methodology.